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(54) Navigation system for an autonomous mobile robot 



(57) A navigation system for an autonomous mobile 
robot within an environment which comprises coded 
signs at predetermined points comprising means for 
storing data regarding said environment, including data 
regarding the positions of said coded signs relative to the 
environment passive vision systems for the image 
acquisition and automatic recognition of said coded 



signs, a computer for estimating its own position and ori- 
entation relative to one of said coded signs and to the 
environment means for acquiring the location of target 
positions, means for planning a path to be covered within 
said environment to reach target positions, and means 
for controlling the robot motion based on the stored path 
data. 
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Description 

This invention relates to a navigation system for 
mobile robots. Robot guidance systems are imple- 
mented in many different forms. During its motion within 
an environment the robot is set to move along predeter- 
mined paths. The robot verifies its own position along the 
path being taken by making odometric measurements. 
This means that sensors positioned at the locomotion 
means, i. e.the wheels, compute the distance travelled by 
the robot and any angular variations in orientation which 
may occur in its trajectory. Helping itself with this type of 
measurement the robot is able to follow any type of tra- 
jectory. However, actually, the slack between moving 
mechanical parts, which cannot be calculated before- 
hand, slippage along the ground, ground roughness and 
other imperfections mean that discordance exist 
between the position calculated at a determined instant 
from the odometric measurements, and the position 
effectively reached at that moment by the robot These 
evaluation errors incrementally add during the move- 
ment of the robot, which can hence at the end of its pre- 
determined path be in a position totally different from that 
required. 

It is therefore necessary to provide the robot with an 
autonomous system for monitoring its position within an 
environment, which whenever the odometricairy esti- 
mated position is different form the effective position 
causes the odometric measurements to be updated to 
their correct values. This provides a cyclic check on the 
real position of the robot together with a trajectory cor- 
rection if a significant error is found. Hence the danger 
of the robot being in an erroneous position at the conclu- 
sion of its path is minimized. 

In the past different ways of making such a check 
have been proposed. One widely practised method is to 
insert into the environment or environments within which 
the robot is supposed to operate, both active and passive 
sensors located in predetermined positions known to the 
robot. Active sensors can be for example active beacons 
emitting coded signals. A passive sensor is located on 
the robot to sense the signals emitted by several bea- 
cons such that the robot navigation system can deduce 
its own position relative to them. The beacons are often 
coupled with RF (radiofrequency) emitters in order to 
transmit the position code associated with the beacon to 
the robot 

Other systems use energy emitting sensors located 
on board the robot, for example rotary laser scanners 
coupled with a photodiode. For this purpose a number 
of retroref lectors are positioned within the environment 
to reflect the signal originating from the rotary emitter. 

The robot navigation system then calculates the 
robot position by suitable triangulation. 

These known systems exhibit drawbacks and prob- 
lems. 

Firstly, as the number of active sensor or reflecting 
objects required for calculating the robot position is 
greater than or equal to three, the number of said sen- 



sors must be very large, so that at every reached position 
in the environment the robot is able to simultaneously 
"see" at least three of said sensors or reflecting objects. 
If the environment is of relatively complex structure, the 
s number of such sensors can become very large. This 
system is therefore particularly invasive of the integrity 
of the operational region. 

In addition, if a laser is used, its irrtoduction into a 
working environment causes safety problems if the emit- 
to ted power exceeds determined levels. Consequently, is 
the emitted power has to be low, the useful operating dis- 
tance becomes low as well, with loss of effectiveness. 

The object of the present invention is to develop a 
navigation system by which the position of the robot 
is within the environment can be estimated in an unambig- 
uous and simple 'manner, without finding the aforesaid 
problems. 

The present invention therefore provides a naviga- 
tion system for an autonomous mobile robot within an 

20 environment which comprises coded signs at predeter- 
mined points, said system comprising means for storing 
data regarding said environment including data regard- 
ing the positions of said coded signs within the environ- 
ment means for the image acquisition and automatic 

25 recognition of said coded signs, a computer for estimat- 
ing its own position and orientation relative to one of said 
coded signs and to the environment, means for acquiring 
the location of target positions, means for planning a path 
to be covered within said environment to reach target 

so positions, and means for controlling the robot movement 
starting with said path data, characterised in that said 
acquisition means are based on a passive visual system 
with standard CCD cameras without requiring any par- 
ticular illumination. 

35 Said coded signs are generally called landmarks 
and are geometric entities, drawn on paper or cardboard 
fixed to the wall or ground, and may have different geo- 
metrical shape according to operative constraints. In par- 
ticular, advantageously, a landmark is adopted, which is 

40 composed of two concentric circles, of which the respec- 
tive diameter is a fundamental characteristic. To increase 
the contrast and visibility to the passive visual system, 
the central region is white whereas the circular ring 
between the two circles is of a dark colour, in particular 

45 black. 

The advantages of using such landmarks are vari- 
ous and considerable. Firstly, there is a closed mathe- 
matical procedure enabling the spatial position and 
orientation of a circle to be calculated (knowing its radius) 

so starting with the equation of the ellipse represented by 
its projection onto the image plane. A double circle, or a 
high-contrast circular ring, generates a very characteris- 
tic image which is difficult to confuse with pictorial struc- 
tures within the environment Finally, compared with 

55 other point configurations usable as landmarks, a circle 
causes no correspondence problems, in that all the 
points in its image ellipse can be used indifferently to esti- 
mate its equation. 
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To make it possibile to distinguish between several 
different signs having the same geometric shape config- 
urations, the ratio of the two diameters is varied while 
maintaining the diameter of the outer circle constant In 
this way, a distintive feature of each sign is given by its 
ratio. 

These landmarks are easily distinguishable from 
each other based on the different diameter of the inner 
circle, but in particular they are easily distinguishable 
from any object which may be located within the environ- 
ment. This is essential to set the robot in a condition 
whereby any confusion and error is avoided. A robot 
should not confuse a different object for a landmark. 
Such landmarks are easy to construct, and because of 
their shape, there are no vertical or horizontal axes along 
which they have to be oriented. 

Preferably the passive vision system is composed 
of a camera mounted on the body of the robot, said cam- 
era being very preferably a black/white CCD camera, 
which generates a standard CCIR video signal. 

The image acquisition means further comprise a 
microcomputer, a system memory, a system bus with 
communication devices, and an image acquisition card 
connected to the microcomputer via the system bus, said 
card sampling a black/white standard CCIR video signal 
and storing it in a memory bank in the form of an image 
matrix 

Further advantages, aspects and characteristics of 
the present invention will be more apparent from the 
description given hereinafter by way of example only. It 
is based on the accompanying figures, in which: 

Figure 1 is a schematic representation of a global 
■ map suitable for the present invention; 
Figure 2 represents a physical map of an environ- 
ment within which an autonomous robot moving in 
accordance with the navigation system of the 
present invention can be used; 
Figure 3 represents an image used for calibrating 
the camera of the present invention; 
Figure 4 shows the relationship between the image 
plane of the camera and its optical centre according 
to the present invention; 

Figure 5 shows the relationship between the camera 
calibration plane and the camera orientation accord- 
ing to the present invention; 
Figure 6 is a flow diagram schematically showing a 
procedure for recognizing a landmark in an image 
according to the present invention; 
Figure 7 is a flow diagram schematically showing the 
steps involved in calibrating a camera according to 
the present invention. 

The navigation system according to the present 
invention must be fed with information concerning the 
configuration of the environment or environments within 
which the robot is supposed to operate. 

This is done by inserting a map known as a global 
map containing the following data: 



graph of reference systems 
graph of navigation points 
list of landmarks. 

5 These data are processed offline during the instal- 
lation phase preceding the effective use of the robot. 

The reference system graph describes the clock- 
wise cartesian reference system on the navigation plan 
(floor) and the coordinate transformations between 

10 them. Each reference system is represented by a vertex 
on the graph whereas the transformation of coordinates 
from one system to another is represented by an arch 
oriented from the vertex associated with the first system 
to that associated with the second and having as 

15 attributes the origin and orientation of the first reference 
system expressed within the second reference system. 

Figure 1 describes the contents of a global map, in 
particular the graphs of the reference systems and nav- 
igation points. 

20 The fist of landmarks contains all the landmarks usa- 
ble for navigation. For each of them, the associated ref- 
erence system, the measurement of the circle diameter 
(i.e. of the outer concentric figure) and the ratio of the 
characteristic corresponding measurements of the two 

25 concentric geometric figures are reported. As already 
stated, this ratio must be such as to unambiguously iden- 
tify the landmark. By convention, the reference system 
associated with the landmark finds its origin in the pro- 
jection of the landmark centre onto the navigation plane 

30 and its y axis corresponding to the straight line normal 
to the landmark plane, the positive direction being that 
emerging from the visible side. The navigation point 
graph contains those points in the navigation plane to 
which the robot vehicle can go and the straight lines 

35 between them along which it can travel. Each navigation 
point is represented on the graph by a vertex having as 
attributes the associated reference system and the point 
position expressed in the associated reference system. 
Figure 2 shows a physical map to be associated with 

40 a global map such as that of Figure 1 . 

In the movement of a robot within an environment, 
the so-called mission is of considerable importance A 
mission is a sequence of goals, which the robot vehicle 
has to reach. Each goal is achieved by executing a list 

4S of primitive perceptive of motory actions, known as a 
task. 

The robot by means of its control system, translates 
into effective action the sequential Gnking of such tasks, 
which constitute the given mission. 
so The implementation of a path or mission, starting 
from its starting point, comprises essentially the following 
steps: 

1- the robot is made to move along the navigation 
55 point graph to the next navigation point: 

2- by odometric measurements the robot evaluates 
its position and stops when the predetermined nav- 
igation point is presumed to have been reached; 
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3- at this point the robot attempts to frame a land- 
mark positioned in proximity to the reached naviga- 
tion point; 

4- having framed the landmark and recognized it 
from the landmark list available to the robot, it esti- 
mates its position and orientation relative to it ami 
on the basis of this information updates its position 
and orientation within the known global map; 

5- the robot repeats steps 1 to 4 for the next naviga- 
tion point until it has completed its mission. 

The essential point of the present invention is the 
method of framing and recognizing the landmark and of 
estimating tie position and orientation of the robot rela- 
tive to the landmark. Of fundamental importance is there- 
fore the ability of the camera to build up an image and to 
obtain geometric information therefrom. 

For a given framed scene, the vision system (com- 
posed of the camera and acquisition card) provides as 
output a bidimensional spatial representation known as 
the image matrix, or merely the image, in the form of a 
rectangular matrix of positive values. It is formed by using 
inside the camera a sensor divided into a rectangular grid 
of photosensitive cells, for each of which the incident light 
intensity is measured. With each of these cells there cor- 
responds a different element of the image matrix, known 
as a pixel, the value of which, known as the grey level, 
is proportional to the light intensity incident on the corre- 
sponding cell. The CCD sensor is a rectangular matrix 
of photosensitive cells mapped onto a corresponding 
pixel "image* matrix obtained by analog/digital sampling 
of the camera output analog signal. To descrfce the geo- 
metric model which relates a generic point within the 
scene to its corresponding pixel within the image its is 
firstly necessary to define as the image plane that plane 
within space which contains the telecamera sensor. As 
can be seen from Figure 4, the image of a generic scene 
point P on the sensor is the point IP intersected on the 
image plane SEN by the straight line joining said point P 
to a f ixed point OC known as the optical centre or pinhole. 
The optical centre is approximately at the centre of the 
camera lens system. As the position of the optical centre 
OC relative to the image plane SEN does not depend 
either on the orientation of the camera or its position, the 
image centre 01 can bedef ined as the perpendicular pro- 
jection of the optical centre OC on the image plane SEN. 
The distance between the optical centre OC and the 
image centre Ol is known as the focal length, and the 
straight line joining said two points is known as the optical 
axis. An image formation model of this type is known as 
a pinhole camera model. The cooixfinates xI(IP) and 
yl(IP) of the image point IP expressed as units of focal 
length within the reference system centered on the 
image centre 01 are obtained from the coordinates xC(P) 
and yC{P) of the point P within the reference system cen- 
tered on OC as: 



x/( /P) = ZOO. . 
X!(in zC(P) 1 



As can be seen from Figure 5, the orientation of the cam- 
era relative to the horizontal navigation plane is defined 

10 by two angles: the tilt angle 6 indicates the inclination of 
the optical axis zC to the navigation plane; the swing 
angle ¥ indicates the rotation between any straight line 
parallel to the horizontal line and the horizontal axis xC 
of the camera. The position of the camera relative to the 

is navigation plane is specified by the height h in millime- 
tres of the optical centre OC above the navigation plane. 

The flow diagram of Figure 6 illustrates the detecting 
pitxiedureof the presence of a landmark within the visual 
field of the camera. The image is divided into a prede- 

20 fined set of equidistant rows each of which is scanned 
horizontally in a search for points in which the light inten- 
sity undergoes a value change exceeding a given thresh- 
old. These points are then considered as the ends of 
segments along which the light intensity is virtually con- 

25 stant Each segment in which the intensity increases in 
moving from the interior of the segment to the outisde is 
considered as black (B) whereas the reverse case is con- 
sidered as white (W). Each line hence generates 
sequences of black segments and white segments. 

30 The same image is then scanned by vertical col- 
umns in the same manner. Again in this case sequences 
of black segments and white segments are obtained. 

By suitably analyzing and reprocessing said seg- 
ment sequences, the form of the possibly framed land- 

35 mark can be constructed and compared with each of the 
possible forms known to the processor. By a first approx- 
imated procedure of intersection of the sequences 
obtained by scanning the image vertically and horizon- 
tally, a rough estimate of the significant dimensions of 

40 the possibly framed landmark is obtained. A more refined 
procedure, in which starting with said rough estimate the 
light intensity gradient in the transition regions from white 
segment to blacksegment or vice versa is analyzed, ena- 
ble the sizes of the two ellipses represented by the image 

45 of the landmark circles to be computed with precision. 
With the more refined procedure, in which starting 
with said rough estimate the light intensity gradient in the 
transition regions from white segment to black segment 
or vice versa is analyzed, it is possible to identify with 

so precision a plurality of points belonging to the images of 
both the circular contours of the landmark. These con- 
tour images are a pair of concentric ellipses, whose 
equations are obtained by fitting the general ellipse 
equation with said points. Starting from the equation of 

55 the outer ellipse and using a mathematical procedure 
known as perspective inversion, the position and orien- 
tation of the landmark relative to the camera is evaluated. 
Such evaluation is done by calculating in which position 
the landmark should be, in order for its outer circular con- 
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tour to produce the acquired image. This corresponds to 
a hypothetical superimposition of the deformed land- 
mark contour and the acquired image. From a knowledge 
of how the camera is positioned and oriented relative to 
the robot it is hence possibile to evaluate the position < 
and orientation of the robot relative to the landmark. The 
same perspective inversion procedure is the repeated on 
the inner ellipse by calculating the distance from the opti- 
cal centre of the camera at which the centre of a circular 
contour of diameter equal to the outer circular contour of 
the landmark would have necessarily to lie in order to 
produce said inner ellipse. 

The ratio of the diameter of the outer circle to the 
diameter of the inner circle of the landmark is then cal- 
culated as the ratio of said distance to the distance of the 
landmark centre from the optical centre as deduced from 
the previously evaluated position and orientation. 

This diameter ratio, indicated in the landmark list as 
a previously known ratio of two corresponding measure- 
ments characteristic of the two concentric circular con- 
tours of the framed landmark, enables the landmark to 
be unambiguously identif ied. 

A fundamental requirement of the system is to have 
calibrated cameras. A very advantageous aspect of the 
present invention is the manner of caltorating the cam- 
era/robot system. In this respect, to be able to obtain 
geometric information on the position and orientation of 
the robot vehicle from images acquired by a vision sys- 
tem on board, it is necessary to have initially identified 
the relationship linking the position relative to the vehicle 
of a generic point within scene ad its image point. To 
achieve this, use is made of a sequence of images of a 
reference object of known characteristics take from dif- 
ferent viewing positions. In particular, a known regular 
rectangular planar lattice formed from a plurality of rows 
and columns of regular geometric shapes is framed. 

With reference to Figure 3. for this purpose a regular 
rectangular planar lattice formed from rows and columns 
of dark identical dots on a fight background is used 
according to the present invention. It is arranged in a 
plane perpendicular to the navigation plane, to define a 
rigidly connected clockwise reference system G having 
as its axis zQ the upwardly directed vertical axis passing 
through the centres of the dots of the central column and 
further having as its origin OG the intersection of zG with 
the navigation plane and as its axis yG the axis parallel 
to the navigation plane and perpendicular to the lattice 
plane directed as the exit vector from the viewing point 
OC and entering the lattice plane perpendicularly. The 
axis xG is the horizontal and lies in the lattice plane 
directed towards the right observing the lattice trontally. 

The purpose of the calibration procedure is to find 
the effective geometric relationship between points 
within the scene and their image points acquired by the 
vision system and the transformation between this latter 
and the odometric measurement system. Using the pin- 
hole camera model, an image formation model must first 
be devised which depends only on a limited number of 
free parameters for which those values must then be 



found such that the model deriving from them repro- 
duces as accurately as possible the transformation from 
three-dimensional points of known position of the current 
odometric reference system to the corresponding points 

? in the image. The free parameters of the image formation 
model are those referred to the pinhole camera model: 
the focal length a u and expressed in width and height 
of a pixBl: the coordinates (uo,v 0 ) of the image centre 
expressed in pixels, the position of the camera relative 

ro to the navigation plane; the height h of the optical centre 
above the navigation plane; the tilt angle 6 and the swing 
angle Other parameters link the camera reference 
system projected onto the navigation plane to the odo- 
metric reference system V rigidly connected to the vehi- 

is cle. Finally, there are those parameters which link the 
initial odometric reference system O with the reference 
system G ridigly connected to the calibration lattice; 
excluding this latter parameters relative to the lattice, the 
values of all the other parameters constitute the result of 

20 the calibration- procedure and are used as input data, 
coming from the landmark location procedures. 

From the image formation model two mathematical 
functions are defined giving the position in pixels of the 
centre of a given dot within the image taken by the robot 

25 vehicle in a certain position and orientation O with 
respect to the initial odometric reference system, the dot 
having known coordinates expressed in the lattice refer- 
ence system, as a function of the model free parameter 
vector. 

30 The calibration procedure is divided into three steps: 
initial image acquisition from various positions, measure- 
ment of the centres of the dots visible in the given 
images, followed by their processing to find the free 
parameters of the image formation model. 
35 In the first step, i.e. image acquisition, use is made 
of a predefined list of a given number of positions and 
orientations which the robot must sequentially reach in 
order to acquire afterwards in each case an image of its 
calibration lattice, which is stored in the image list for the 
40 second part of the calibration procedure. 

Associated with each acquired image, the position 
and orientation of the robot estimated by the odometric 
system within a reference system whch does not vary 
during the procedure is stored. In the second step the 
as user considers all the images of the image list prepared 
during the first step. In each of these he selects four not 
aligned in triplets reference dots. The transformation 
from points of the lattice to corresponding points of the 
image is a transformation between two projecting planes, 
so so that it is possible to interpolate the positions of the 
centres of the four selected dots to also estimate the cen- 
tres of the other lattice dots on the image. The image is 
binarized, i.e. the grey level of each pixel is set to the 
maximum or minimum value possible according to 
55 whether it is greater or less than a certain threshold set 
by the user. For each lattice dot a procedure is then per- 
formed for measuring the position of its centre within the 
image. 
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Use is made here of the tact that the pixels of a dot 
image are ail at the same value by the effect of the bina- 
rization, as the lattice dots are dark on a clear back- 
ground The maximum connected region having the 
maximum grey level formed from pixels and containing 5 
that pixel which the preceding interpolation indicated as 
the dot centre is then identified. The new measurement 
of the centre position is then calculated as the position 
of the barycentre of the pixels of this region. 

In the third calibration step the predicted position of 10 
the centres of the framed dots in the various images is 
calculated on the basis of the free parameters of the 
image formation model to hence construct an error func- 
tion which is the sum of squares of the distances of the 
positions of the dot centre on the image measured from is 
their positions as predicted by the image formation 
model when the vehicle is in the corresponding position 
and orientation specified in the position list for image tak- 
ing. 

The free parameter values which minimize this error 20 
function are those which best fit real images. 

It should be noted that according to the present 
invention, the robot in order to correctly identify its posi- 
tion needs to frame only one landmark, and not at least 
three as in the state of the art. This means that the nav- 2s 
igation system is less invasive of the environment is of 
lower cost and requires less maintenance. During the 
framing procedure it may happen that the robot has 
accumulated such a quantity of position errors that it is 
unable to find any landmark within the expected field of 30 
vision. In this case provision is made to rotate the camera 
in a horizontal plane. This facility increases the reliability 
of the entire robot, and allows it to be virtually perfectly 
oriented under all conditions within the environment in 
which it moves. For improved handling and control, the 35 
robot further comprises an interactive terminal. Said 
interactive terminal comprises an alphanumerical dis- 
play and a keyboard. 

Claims " 

1 . A navigation system for an autonomous mobile robot 
within an environment which comprises coded signs 
at predetermined points, said system comprising 
means for storing data regarding said environment, 4S 
including data regarding the positions of said coded 
signs within the environment means tor the image 
acquisition and automatic recognition of said coded 
signs, a computer tor estimating its own position and 
orientation relative to one of said coded signs and so 
to the environment means tor acquiring the location 
of target positions, means tor planning a path to be 
covered within said environment to reach target 
positions, and means tor controlling the robot move- 
ment starting with said path data, characterised in ss 
that said acquisition means are based on a passive 
visual system with standard CCD cameras without 
requiring any particular illumination. 



2. A system as claimed in claim 1 , characterised in that 
said coded signs are geometric entities composed 
of one outer and one inner concentric circles, each 
of these two circles having a characteristic dimen- 
sion, i.e. its diameter. 

3. A system as claimed in claim 2, characterised in that 
the ratio of the two diameters of the two concentric 
circles is variable, the diameter of the outer circle 
remaining constant 

4. A system as claimed in claim 1 , characterised in that 
said passive vision system is composed of a camera 
mounted rigidly on the body of the robot. 

5. A system as claimed in claim 4, characterised in that 
said camera is a black/white CCD camera generat- 
ing a standard CC1R video signal. 

6. A system as claimed in claim 1 , characterised in that 
said computer for estimating the position and orien- 
tation of the robot with respect to the relative coded 
sign uses perspective inversion of the ellipse equa- 
tion which comes from the recording of said coded 
sign executed by said passive vision system. 

7. A system as claimed in claim 1 , characterised in that 
said image acquisition means comprise a micro- 
computer, a system memory, a system bus with 
communication devices, and an image acquisition 
card connected to the microcomputer via the system 
bus, said card sampling a black/white standard 
CCIR video signal and storing it in a memory bank 
in the form of an image matrix. 

8. A system as claimed in claim 1, characterised in that 
said robot further comprises an interactive terminal. 

9. A system as claimed in claim 8, characterised in that 
said interactive terminal is composed of an alphanu- 
merical display and a keyboard. 

10. A robot provided with the navigation system of claim 
1, for which an installation procedure is provided 
consisting of the following points: 

a) calibration of the robot/camera system; 

b) acquisition and relative storage of data 
regarding the environment within which the path 
is to be covered; after which, when positioned 
in proximity to a coded sign, 

c) the user provides the robot with the target 
position to be reached, or the end-of-mission 
command; 

d) the robot plans the path to be taken starting 
from the target position to be reached and the 
data regarding the environment by dividing it 
into successive points to be reached; 
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e) moving the robot by said movement control 
means to the next point of the planned path; 

f) framing a coded sign by said passive vision 
system; 

g) recognition of the coded sign; 5 

h) estimation of its own position and orientation 
relative to said coded sign; 

i) calculating its own position and orientation rel- 
ative to the environment starting with the posi- 
tion and orientation relative to the coded sign; 10 

■ j) controlled movement by said movement con- 
trol means to the next point on the planned path 
starting from the position calculated under point 

i; ■ ' 

k) repetition of points e. f, g. h, i and j until com- is 

pletion of the planned points on the path; 

I) repetition of point c, until the end-of-mission 

command. 

1 1 . A robot as claimed in claim 1 0, characterised in that 20 
the calibration of point a) comprises the following 
steps: 

- an initial image acquisition by framing from var- 
ious positions a known regular rectangular 25 
planer lattice formed from a plurality of rows and 
columns of regular geometric shapes, such as 
dark identical dots on a light background; 
storing the robot position and orientation esti- 
mated by the automatic system within an invar- 30 
iant reference system. 

measuring the positions of the centres of the fig- 
ures visible in the data images, 
processing said positions and said robot posi- 
tion and orientation to find the free parameters 35 
of the image formation model. 
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